Estimating Node Similarity by Sampling Streaming Bipartite Graphs

Bipartite graph data increasingly occurs as a stream of edges that represent transactions, e.g., purchases by retail customers. Applications such as recommender systems employ neighborhood-based measures of node similarity, such as the pairwise number of common neighbors (CN) and related metrics. While the number of node pairs that share neighbors is potentially enormous, in real-word graphs only a relatively small proportion of all pairs have a large number of common neighbors. This motivates finding a weighted sampling approach that preferentially samples such node pairs. This paper presents a new sampling algorithm that provides a fixed size unbiased estimate of the similarity (or projected) graph on a bipartite edge stream. The algorithm has two components. First, it maintains a reservoir of sampled bipartite edges with sampling weights that favor selection of high similarity nodes. Second, arriving edges generate a stream of similarity updates based on their adjacency with the current sample. These updates are aggregated in a second reservoir sample-based stream aggregator to yield the final unbiased estimate. Experiments on real world graphs show that a 10% sample ar each stages yields estimates of high similarity edges with weighted relative errors of about 10−2.